Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement v1 file format #299

Merged
merged 329 commits into from
Jan 31, 2025
Merged

Implement v1 file format #299

merged 329 commits into from
Jan 31, 2025

Conversation

t7phy
Copy link
Member

@t7phy t7phy commented Jul 12, 2024

Feature TODOs:

  • Generalize Channel to support arbitrarily many PIDs. This is implemented with commits f32902a, 2417c92 and 03f2dc6.
  • The structure Order should support a fragmentation scale. Started with commits 5e281ae and 02e48e9.
  • The CLI should support varying this new fragmentation scale. We need the following point variations: 1 (no scale variation), 3 (vary all scales with the same factor), 7 and 9 (assuming that there's no fragmentation scale), and 17 and 27 (assuming there's a fragmentation scale)
  • Extend the evolution code to take care of (at least) one, two and three convolutions with possibly one, two and three different EKOs
  • Finally, we need new subgrid types that support possibly arbitrarily many scales and momentum fractions and which support filling or are optimized for space. We should use the PackedArray struct that we wrote some time ago.
  • To fix the tests, we need import v0 PineAPPL grids and convert the subgrids into the new subgrid types.
  • Let the user specify on the CLI whether are convolution function is polarized or a fragmentation function; we can use a syntax like LHAPDFNAME+p for polarized and LHAPDFNAME+f for fragmentation function and LHAPDFNAME+pf or LHAPDFNAME+fp for both
  • Add machine-readable MC results and uncertainties to calculate interpolation error #270; started with commit 029f3a1

Code TODOs:

  • Change types of members of Order to u8
  • Generalize channel! to arbitrarily many PIDs
  • Support arbitrarily many scales and fix import of flexible-scale fastNLO tables
  • Test evolution for flexible-scale grids with ZEUS_2JET_319GEV_374PB-1_DIF_ETQ2_BIN6 (available in the same location where also the other test data is)
  • Simplify bin treatment: remove BinInfo, and have instead fill limits (1d limits that only concern Grid::fill) and bin limits (n-dimensional limits)
  • Add fragmentation scales to Grid::evolve_info
  • Make it possible to have different initial factorization and fragmentation scales in an FK-table
  • Rename lagrange_subgrid
  • Add support for static scale detection in previous subgrid type
  • Rename PackedQ1X2SubgridV1
  • Remove NodeValues
  • Remove Mu2
  • Implement D=N for general_tensor_mul in evolution.rs
  • In Grid::merge check if grids are compatible with each other before merging
  • in the CLI remove query to SetType:
  • in the CLI don't assume polarization of the convolution functions from the grid
  • Test EvolveInfo::frg1 and FkTable::frg0 @Radonirinaunimi Test remaining new features of PineAPPL 1.0 #325
  • Test different initial factorization and fragmentation scales in an FK-table @Radonirinaunimi Test remaining new features of PineAPPL 1.0 #325
  • Test D=3 for general_tensor_mul in evolution.rs @Radonirinaunimi Test remaining new features of PineAPPL 1.0 #325
  • Fill in all the TODO comments Polish code for PineAPPL 1.0 #326
  • Cleanup code Polish code for PineAPPL 1.0 #326
  • Raise code coverage Polish code for PineAPPL 1.0 #326

CAPI TODO:

  • increase code coverage
  • unbreak GridOptFlags (see commit 54a59f3)
  • rename pineappl_channel_* to pineappl_channels_*?
  • check generated grids of new example programs
  • rename -v1.cpp suffix of example programs to .cpp and rename the previous example programs from .cpp to -deprecated.cpp. This should clarify that the use of v0 functions is deprecated (but still works!)

Fortran module TODO:

  • add new functions

Python API:

  • update to pyo3-0.22.5, this possible now since numpy-0.22.0 was released. Figure out what new features we can leverage from 0.22.x (see its CHANGELOG):
    • numpy-2 support (works out of the box)
    • declarative modules and submodules
    • Python 3.13
    • recheck whether we still need #![allow(unsafe_op_in_unsafe_fn)]
  • raise minimum required Python version to 3.7 in pyproject.toml
  • propagate bin-related changes @Radonirinaunimi
  • Update tutorials Update Python tutorial for PineAPPL 1.0 #327

General:

  • merge CHANGLOG.md entries from v0.8.x branch
  • write a summary of the changes for v1.x

@t7phy t7phy linked an issue Jul 12, 2024 that may be closed by this pull request
15 tasks
@cschwan cschwan linked an issue Jul 31, 2024 that may be closed by this pull request
5 tasks
@cschwan cschwan removed a link to an issue Jul 31, 2024
15 tasks
@cschwan
Copy link
Contributor

cschwan commented Jul 31, 2024

I believe the list shown in #118 is probably a bit too ambitious for this pull request. Realistically, we should implement all the items listed in #172.

@cschwan cschwan changed the title makes the necessary changes for v1 format [WIP] Implement v1 file format [WIP] Aug 6, 2024
pineappl/src/boc.rs Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved

/// TODO
#[must_use]
pub fn fill_index(&self, value: f64) -> Option<usize> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this prefix is a bit confusing, i.e. fill_index and fill_limits are not similar to each other - maybe insert_at_index?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or actually, just insert?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are similar to each other in that they both refer to Grid::fill. This method has a parameter observable which is an f64, which is being checked if it falls inside the limits returned by BinsWithFillLimits::fill_limits. If that's not the case, BinsWithFillLimits::fill_index returns None, else Some(index) where index is the result. Of course, this should be part of the documentation that is still missing.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I misunderstood (so yes documentation) ... mmm I still think it's not a perfect name (as usual this is a subjective thing); how about search_fill_index? if you disagree please just resolve this comment


/// Error type returned by [`BinRemapper::from_str`]
#[derive(Debug, Error)]
pub enum Bla {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
pub enum Bla {
pub enum BinRemapperStrError {

nur um das klar zu stellen: mein Standardname is 50% Bla und 50% Blub 🙈

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that still has to be changed.

@@ -124,17 +124,17 @@ bitflags! {
#[derive(Clone, Deserialize, Serialize)]
pub struct Grid {
subgrids: Array3<SubgridEnum>,
channels: Vec<Channel>,
bin_limits: BinLimits,
bwfl: BinsWithFillLimits,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we call it just bins here please? We don't care about the implementation details inside grid

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not an implementation detail. Every bin has effectively two sets of limits:

  1. The N-dimensional actual bin limits that also the user sees. These mustn't overlap, but they allow 'gaps' between them, also they must all have the same dimensions;
  2. Fill limits: these are one-dimensional limits, the 'fill limits', that only Grid::fill uses. These are as many limits as there are bins plus one more and all values must be ascending.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That being said I suppose we can find a better name. I'd like to avoid Bins though, because this struct has a method fn bins() and then the code would have a lot of grid.bins().bins() invocations, which is a bit odd.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That being said I suppose we can find a better name.

that was my main concern - bins_config?

impl FromStr for BinsWithFillLimits {
type Err = Bla;

fn from_str(s: &str) -> Result<Self, Bla> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to split this looong function into 4-5 smaller ones? naively it seems there is a clear path step 1 -> step 2 -> step 3 -> ... which act in isolation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can split this function into smaller ones, but I don't see that making it more readable. But let me try to improve it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to me it would be more readable - because then there will be a clear input, a clear task, and a clear output. Needless to say that this would also be good for unit testing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that if this could be broken into smaller functions that'd be great as right now it is a bit difficult to read.

@t7phy t7phy merged commit 45afc97 into master Jan 31, 2025
10 checks passed
@t7phy t7phy deleted the v1-file-format branch January 31, 2025 10:12
@t7phy t7phy changed the title Implement v1 file format [WIP] Implement v1 file format Jan 31, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants